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ABSTRACT 


The  mission  requirement  of  low-level,  nighttime  navigation 
employing  night  vision  goggles  has  expanded.  This  has  led  to 
a  greater  demand  for  training  NVG  skills  and  initiated  a 
requirement  for  a  low-cost,  desktop,  computer-based  trainer 
(CBT) . 

A  framework  is  presented  in  this  thesis  that  includes  a 
review  of  the  technology  available  for  designing  a  night 
vision  CBT.  System  attributes  and  constraints  are  identified 
and  analyzed,  and  evaluation  criteria  developed  to  allow  for 
examination  of  alternative  system  configurations. 

Two  configurations  are  developed:  one  PC-based  and  one 
workstation-based.  These  configurations  present  different 
cost/ benefit  components.  A  sample  review  in  graphics 
capabilities,  processor  performance  and  peripheral  support  is 
provided  for  the  two  configurations.  In  addition,  cost  range 
estimates  are  included  and  possible  baseline  capabilities 
established  to  assist  in  the  determination  process. 
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INITIAL  DISTRIBUTION  LIST 


I .  INTRODUCTION 


A.  BASIS  FOR  EVALUATION 

In  order  to  prepare  aircrew  for  the  expanding  role  of 
night  vision  employment  in  fixed-winged  aircraft,  a  low-cost, 
portable,  computer-based  trainer  (CBT)  needs  to  be  developed. 
This  part-task  trainer  will  focus  on  enhancing  aircrew 
proficiency  and  confidence,  attributes  that  directly  relate  to 
improved  combat  effectiveness  and  greater  safety  of  flight 
awareness. 

The  mission  expansion  in  night  vision  has  required  aircrew 
to  develop  a  more  thorough  training  syllabus,  especially  while 
deployed.  A  CBT  that  incorporates  a  representative  head- 
mounted  display  will  enable  aircrew  to  simulate  flying  in  the 
night  vision  environment  and  support  a  tactical  doctrine  that 
requires  a  low-level,  nighttime  strike  capability. 

Designing  an  framework  for  evaluating  the  associated 
computer  technology  is  the  initial  stage  of  the  project.  This 
framework  will  be  utilized  to  assist  in  the  selection  process 
for  the  prescribed  desktop  trainer. 

B .  GENERAL  METHODOLOGY 

This  thesis  will  initially  focus  on  determining  possible 
system  design  requirements.  Certain  limitations  exist,  such 
as  uncertain  user  requirements,  potential  changes  in 
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specifications  and  a  short  developmental  time  frame. 
Evaluation  criteria  will  be  identified  that  could  be  used  to 
assist  in  the  assessment  of  available  computer  technology  and 
head-mounted  displays. 

A  framework  will  be  constructed  using  a  structured 
approach  to  design  the  system  capabilities.  This  framework 
will  be  used  to  identify  and  compare  alternative 
configurations  for  a  CBT  and  ascertain  their  respective 
capabilities  and  costs. 

Finally,  a  practical  example  will  be  presented  to  help  the 
reader  understand  the  possible  trade-offs  between  capabilities 
and  costs. 

C.  SCOPE  OF  THESIS 

This  thesis  will  present  general  discussions  of  candidate 
computing  systems.  Capabilities  across  a  broad  spectrum  will 
be  discussed  and  price  ranges  presented. 

Because  these  specifications  are  subject  to  change  in  the 
highly  dynamic  computer  industry,  estimates  are  intended  to 
give  the  reader  an  overall  view  of  possible  alternatives,  and 
should  not  be  construed  as  definite  figures. 

1.  Research  Objective 

The  objective  of  this  thesis  can  be  stated  as  follows: 

Design  an  evaluation  format  that  will  introduce 
evaluation  criteria  for  possible  alternative  configurations, 
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identify  current  computer  technology,  and  present  a  practical 
application  of  the  established  format  framework. 

D.  BACKGROUND 

Perhaps  the  most  perishable  skill  in  flying  fixed-wing, 
tactical  aircraft  is  nighttime,  low-level,  terrain-following 
navigation,  employing  night  vision  goggles,  at  speeds  in 
excess  of  400  nautical  miles  per  hour  (knots)  .  It  is 
extremely  difficult  to  stay  proficient  in  this  labor  intensive 
environment,  especially  while  deployed.  Loss  of  spatial 
awareness,  poor  flight  discipline  and  airborne  mishaps  are  the 
result  of  inadequate  preparation  and  insufficient  training. 
Lessons  learned  from  participating  aircrew  and  their 
respective  airwing  commanders  during  Operation  Desert  Storm 
confirmed  the  inadequacies  of  current,  established  procedures 
(Ref.  1:  p.II-3].  The  potential  for  loss-of-life  in  both 

training  and  operational  sorties  exists  and  is  clearly 
unacceptable . 

Since  airborne  training  time  aboard  an  aircraft  carrier 
allows  for  only  minimum  attention  to  many  mission 
requirements,  a  CBT,  with  accurate  graphical  representations 
of  real-world  scenarios  and  an  available  interface  to  a  head- 
mounted  display,  would  provide  a  low-cost,  available  and  easy 
to  use  training  tool  to  the  applicable  aircrew. 

This  CBT  would  allow  for  advanced,  realistic  training  of 
a  unique  environment  that  requires  aircrew  performance  to 
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deviate  from  normal  procedures.  Visual  cues  and  spatial 
orientation  while  wearing  night  vision  equipment  are  vastly 
different  from  those  employed  during  routine  operations 
(examples  being  restricted  f ield-of-view  and  lack  of  depth 
perception) ,  not  to  mention  wearing  a  somewhat  cumbersome 
piece  of  head-mounted  hardware,  in  addition  to  the  helmet. 

E.  ORGANIZATION  OF  THE  THESIS 

The  thesis  is  organized  as  follows: 

1.  Chapter  II.  Design  Concerns 

This  chapter  focuses  on  the  requirements  for  designing 
a  CBT  and  the  possible  changes  in  system  specifications. 
Issues  addressed  include  current  simulation  techniques, 
computer  technology  trends  and  the  emphasis  on  off-the-shelf 
procurement.  A  methodology  is  introduced  to: 

1.  Help  identify  specifications  and  constraints. 

2.  Discuss  data  collection  techniques. 

3.  Present  possible  alternative  configurations. 

2.  Chapter  III.  Framework 

In  this  chapter,  an  evaluation  framework  is 
established  and  the  evaluation  criteria  is  defined  and 
discussed.  These  criteria  will  help  clarify  issues  such  as 
system  performance,  peripherals  and  component  capabilities. 
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The  design  of  the  format  follows  a  sequential  approach  with 
each  component  step  discussed. 

3.  Chapter  IV.  Application  of  Framework 

The  chapter  defines  the  functional  components  of  two 
candidate  systems,  a  personal  computer  and  a  workstation. 
These  components  consist  of  peripherals,  the  monitor,  the 
head-mounted  display  and  the  internal  architecture  of  the 
computing  system. 

A  discussion  of  the  evaluation  criteria  is  presented 
and  general  cost  ranges  provided  to  help  quantify  the 
evaluation.  The  component  capabilities  and  costs  are 
formulated  and  presented  in  tabular  form  in  Appendix  A  and  B. 
Although  the  selection  process  is  outside  the  scope  of  this 
thesis,  a  brief  discussion  recommends  possible  avenues  to 
follow  in  the  selection  process. 
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II .  DESIGN  CONCERNS 


A.  INTRODUCTION 

The  design  of  a  CBT  for  training  NVG  skills  presents  some 
interesting  issues.  Since  a  CBT  of  this  kind  has  yet  to  be 
developed,  there  is  little  background  data  to  review. 
Consequently,  uncertain  user  requirements  and  changes  in 
design  specifications  must  be  anticipated.  In  addition,  the 
time  frame  for  development  should  be  relatively  short,  in 
order  to  make  this  system  available  to  operational  users  as 
soon  as  possible. 

l.  uncertain  User  Requirements 

Little  historical  information  exists  to  quantify  the 
user  requirements  for  this  CBT.  Only  through  an  in-depth 
evaluation  of  the  participating  aircrew  can  a  truly 
representative  system  be  constructed.  This  would  involve 
extensive  interview  and  survey  of  aircrew  currently  flying 
fixed-wing  aircraft  such  as  the  F/A-18,  the  AV-8B  and  the  A-6. 
These  aircraft  are  night  vision  compatible  and  the  respective 
aircrew  are  the  most  qualified  to  make  valued  inputs. 
However,  compiling  these  inputs  into  a  coordinated  list  of 
requirements  is  both  time  and  labor  intensive.  Consequently, 
until  the  user  requirements  can  be  defined,  the  design 
requirements  of  a  CBT  must  be  at  a  high  level,  at  least 
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initially,  to  allow  for  redesign  to  better  meet  user  demands. 

2.  Anticipated  Change  in  Specifications 

Although  system  specifications  have  been  established 
(they  will  be  discussed  later  in  the  chapter) ,  they  are  still 
flexible,  until  they  can  be  applied  to  the  operational 
environment.  Once  applied,  changes  to  these  specifications 
must  be  anticipated  to  minimize  avoidable  delays  in 
development . 

3.  Short  Developmental  Time  Frame 

In  order  to  meet  the  requirement  for  a  desktop  trainer 
to  supplement  current  aircrew  training,  this  system  must  be 
developed  in  a  short  time  frame.  Because  of  the  expansion  of 
the  mission  requirement  for  night-vision,  operational  aircrew 
must  immediately  supplement  their  training  syllabus. 

Before  pursuing  the  actual  design  methodology,  it  is 
important  to  discuss  current  simulation  techniques,  computer 
technology  trends  and  the  emphasis  on  commercial  off-the-shelf 
procurement . 

B.  CURRENT  SIMULATION  TECHNIQUES 
l.  Large  Scale  Simulators 

Airborne  simulation  has  traditionally  been  conducted 
using  large  scale  simulators,  normally  incorporating  complete 
cockpit  replication.  These  simulators  are  extremely  expensive 
and  take  several  years  to  develop  and  implement.  Most  utilize 
an  entire  building  to  house  the  cockpit,  the  simulation 
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environment  (usually  in  the  form  of  a  domed  room)  ,  and  a 
mainframe  computer  to  drive  the  simulation  software.  These 
systems  provide  aircrew  with  realistic  representations  of 
different  airborne  environments,  as  well  as  real-time  inputs 
from  an  operator  console.  They  are  designed  primarily  for 
initial  training  of  aircrew,  with  emphasis  on  proper  cockpit 
and  airborne  discipline.  The  secondary  mission  is  fleet 
refresher  training  for  operational  squadrons. 

Obviously,  the  support  requirements  are  enormous,  with 
full-time  personnel  staffed  for  daily  operation.  Due  to 
upgrade  requirements,  there  is  a  certain  amount  of  down-time 
inherent  in  these  systems.  This  down-time  can  range  from  days 
to  weeks,  depending  on  the  system. 

C.  COMPUTER  TECHNOLOGY  TRENDS 

1.  Reduced  Distinction  Between  PC  and  Workstation 

The  advancements  in  computer  technology  have  allowed 
the  personal  computer  to  accelerate  its  performance 
capabilities. 

Many  workstations,  prior  to  two  years  ago,  had  a 
performance  advantage  over  personal  computers  because  of  a 
superior  central  processor  unit  (CPU) .  However,  the  Intel  486 
chip  has  provided  clock  speeds  of  66  MHz,  which  is  equal  to 
many  popular  workstations,  and  superior  to  some. 

Overall  system  performance  cannot  be  measured  by  clock 
speeds  alone,  but  it  helps  to  demonstrate  that  the  performance 
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gap  between  the  two  classes  of  computers  is  narrower  than 
ever  before. 

2.  Dynamic  Environment 

Computer  technology  continues  to  change  so  quickly 
that  it  is  often  difficult  to  remain  current  on  many  of  the 
latest  innovations.  Capabilities  continue  to  increase,  almost 
exponentially,  while  cost  and  system  size  decrease. 
Information  gathering  has  evolved  into  a  science,  with 
commercial  publications,  trade  journals  and  technical 
publications  inundating  the  computer  market.  It  is  incumbent 
on  any  researcher  to  survey  as  many  sources  as  possible  in 
order  to  make  quality  decisions  about  computing  products. 

3.  Improvements  in  Graphics  Technology 

Dedicated  graphics  processors,  known  as  accelerators, 
have  relieved  much  of  the  graphics  processing  burden  on  the 
CPU  and  allowed  for  24-bit  (16.7  million  colors)  graphics 
operation.  Software  reconfiguration  of  the  screen  resolution 
allows  for  full  use  of  the  processor's  available  memory, 
normally  between  one  and  two  megabytes.  This  provides 
graphics  resolution  of  up  to  1600  x  1200  pixels.  [Ref.  2:p.40] 

D.  EMPHASIS  ON  COMMERCIAL  OFF-THE-SHELF  PROCUREMENT 

Their  is  currently  great  emphasis  within  DoD  on  increasing 
the  use  of  commercial  off-the-shelf  (COTS)  products.  These 
products  require  very  little  government  development  and  are 
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normally  very  accessible.  Some  principal  advantages  of  this 
approach  are  the  following: 


•  Research  and  development  costs  are  reduced,  sometimes 
eliminated. 

•  Current  technology  can  be  immediately  procured. 

•  User  requirements,  once  identified,  can  be  quickly 
satisfied. 

•  Development  and  implementation  time  is  greatly  reduced. 

•  Increased  competition  from  multiple  sources  can  cause 
reductions  in  costs.  [Ref.  3:  p.22] 


There  are  disadvantages  to  this  procurement  policy  and 
they  include: 


•  Commercially-procured  items  often  must  be  assigned  a 
MILSPEC  number,  noting  that  it  meets  military 
specifications  for  reliability  and  safety. 

•  Logistical  support  issues  and  training  can  cause  increases 
in  life  cycle  costs. 

•  Tradeoffs  in  cost  and  performance  is  sometimes  inevitable 
in  order  to  accommodate  commercial  components.  [Ref.  3: 
p.23] 


Many  of  the  disadvantages  can  be  offset  with  proper  life 
cycle  management  and  through  commonality  of  components. 
Planning  for  delays  in  inspection  and  for  integration  of 
training  requirements  for  similar  system  elements  will  reduce 
the  administrative  burden  of  off-the-shelf  procurement. 


10 


B.  METHODOLOGY 


In  order  to  address  the  design  concerns,  a  systematic 
approach  was  utilized. 

Initially,  overall  system  specifications,  including 
constraints,  were  developed. 

Next,  a  survey  of  available  resources  was  conducted,  using 
data  collection  from  various  sources.  This  provided 
information  on  currently  available  commercial  equipment  in  the 
area  of  desktop  simulation  and  identified  the  criteria 
critical  to  the  evaluation  of  a  CBT. 

Finally,  alternative  configurations  were  designed  to  allow 
for  evaluation  and  comparison  in  price  and  performance.  This 
step  is  also  the  precursor  to  the  selection  and  construction 
phases  in  traditional  prototype  life  cycle  development.  [Ref. 
4 :  p. 15 ] 

This  technique  was  preferred  because  of  the  previously 
discussed  issues  of  uncertain  user  requirements,  possible 
specification  changes  and  the  need  to  be  operational  in  a 
short  time  frame. 

1.  System  Specifications 

Four  specifications  were  identified.  They  are 
realistic  graphical  display,  development  of  a  night  vision 
database,  compatibility  with  head-mounted  displays,  and 
deployability  of  the  system. 
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a.  Realistic  Graphical  Display 

The  graphics  presented  should  be  as  even  and  fluid 
as  possible  to  provide  realistic  simulation  for  the  aircrew. 
The  display  must  be  refreshed  in  such  a  way  that  the  aircrew 
can  respond  to  the  simulated  visual  cues  of  the  landscape. 

A  problem  presents  itself  in  this  area.  What  is 
representative?  Is  full-motion  video  required  or  would 
something  less  fluid  provide  acceptable  training.  The  only 
way  to  determine  acceptability  would  be  to  directly  survey  the 
users,  allowing  them  to  view  different  presentations  and 
measure  them  against  actual  operational  sorties.  For  the 
purpose  of  this  thesis,  generally  accepted  baseline  values 
will  be  discussed  in  the  next  chapter. 

b.  Development  of  a  Night  Vision  Database 

This  database  will  contain  graphical  presentations 
consisting  of  digitized  data  from  various  sources.  The 
construction  of  these  presentations  is  demanding  and 
laborious.  If  a  representative  database  is  not  available  in 
time  for  system  development,  a  video  input  source,  such  as 
full-motion  video,  would  be  substituted. 

(1)  Difficulties  in  Production.  Very  few,  if  any, 
night-vision  databases  have  been  produced.  Reasons  include: 

•  Accurate  shading  representations  are  very  complex. 
Ambient  light  conditions  are  governed  by  atmospheric 
conditions  (clouds,  fog,  haze)  and  lunar  stages  (full, 
half ,  etc. . . )  . 
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•  Blooming  (white  sparkles)  is  hard  to  simulate. 

•  The  generation  of  the  depth  perception  limitation,  due  to 
the  loss  of  visual  cues,  requires  the  employment  of 
strenuous  techniques  for  accurate  presentation. 

•  Creating  representative  terrain  maps  are  extremely  time 
intensive,  even  using  developmental  tools  that  can  capture 
digital  mapping  data.  Converting  the  data  into  different 
shades  of  green  significantly  delays  database 
conf iguration . 


c.  Compatibility  with  Head-Mounted  Displays 

The  technical  capability  of  available  computing 
hardware  might  alleviate  the  requirement  of  training  while 
wearing  a  head-mounted  display.  Many  systems  might  provide 
the  resolution  and  the  graphical  representation  required,  on 
a  monitor,  but  realism  would  be  sacrificed.  The  training 
issues  of  restricted  f ield-of-view  (FOV)  and  reduced  depth 
perception  can  only  be  addressed  by  using  a  head-mounted 
display. 

d.  Deployability 

One  of  the  biggest  factors  in  the  system's 
requirements  involved  the  ability  to  deploy  the  system  onboard 
an  at-sea  aircraft  carrier  or  perhaps  an  airwing  on 
detachment.  The  CBT  must  be  available  to  the  aircrew  when 
they  are  not  at  their  home  base  and  that  requires  a  system 
that  is  compact  and  rugged  enough  to  withstand  frequent 
transportation . 
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e.  Constraints 


Three  constraints  are  evident.  The  first,  and 
probably  most  prohibitive,  is  cost.  DoD  procurement  policies 
require  close  scrutiny  of  cost  factors.  The  other  two 
constraints  are  timeliness  of  final  system  delivery  and 
system  complexity. 

(1)  Costs.  A  shrinking  military  budget  mandates 
that  procurement  policies  aim  for  the  most  value/ performance 
for  each  dollar  spent.  Replacing  large  legacy  programs  with 
smaller,  cheaper,  operational-specific  systems  are 
underscored.  Consequently,  this  system  must  meet  those 
congressionally- legislated  requirements. 

Any  procurement  decision  must  analyze  incurred 
costs  in  areas  of  bulk  versus  single  purchase,  available 
government  furnished  equipment  (GFE)  versus  independent 
contract  and  price  discounts  for  governmental  use. 

(2)  System  Delivery .  The  mission  requirement  of 
low-level  flight  using  NVGs  will  continue  to  expand  and  the 
training  and  proficiency  of  the  aircrew  must  augment  this 
expansion. 

Undue  delays  in  the  areas  of  analysis, 
procurement  and  development  must  not  manifest  themselves, 
because  the  trade-off  could  be  human  lives.  As  previously 
discussed,  proper  life  cycle  management  would  help  prevent 
possible  delivery  slippage. 
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(3)  System  Complexity .  The  system  must  be  easy  to 
use.  Users  must  be  integrated  in  the  development  phase  to 
ensure  that  simplicity  is  provided.  Tutorials,  both  software 
driven  and  in  written  text,  must  be  included  in  the  final 
delivery.  System  training,  provided  by  appropriate  vendors, 
can  provide  expertise  and  furnish  much  needed  proficiency. 

2.  Survey  and  Data  Collection 

Collecting  the  information  on  this  project  involved 
conducting  direct  interviews  with  marketing  representatives 
from  computer  companies  and  published  specialist  in  the  field 
of  desk-top  simulation,  reviewing  computer  periodicals  and 
directly  interacting  with  technical  experts, 
a .  Interviews 

Consultation  with  representatives  from  Evans  and 
Sutherland,  Silicon  Graphics  and  Kaiser  Electro-Optics 
provided  invaluable  insight  into  system  development.  Both 
hardware  and  software  requirements  were  discussed  and 
alternatives  identified.  Procurement  costs  were  also 
supplied. 

Essential  information  on  system  component 
construction,  possible  database  requirements  and  a  list  of 
features  for  desktop  simulation  applications  was  also  obtained 
through  the  interview  process.  [Ref.  5:  p.486] 
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b.  Computer  Periodical  Review 

Computer  periodicals,  especially  Imaging  World , 
Silicon  Graphics  World,  PC  Magazine  and  Computerworld ,  are  a 
valuable  source  of  current  computer  technology.  Discussions 
in  the  domains  of  hardware  comparisons,  software  development 
and  graphics  presentation  proved  enlightening  and  are  a 
valuable  basis  for  evaluating  system  requirements. 

c.  Consultation  with  Technical  Experts 

Discussions  on  head-mounted  displays  were  conducted 

with  technical  representatives  of  Optics  One  and  Kaiser 
Electro-Optics . 

A  demonstration  of  the  Sim-Eye  head-mounted  display 
from  Kaiser  enabled  a  first-hand  evaluation  of  its 
capabilities  in  the  areas  of  FOV  and  focus  range.  It  also 
provided  a  user’s  prospective  on  size,  weight  and  durability 
of  the  head-mounted  display. 

3.  Alternative  Hardware  Configurations 

Analyzing  different  system  configurations  provide 
qualitative  measurements  of  overall  performance  capabilities 
and  physical  system  set-up  and  storage  requirements. 

Designing  alternative  configurations  will  enable 
direct  input  and  feedback  from  designers  and  users.  Hard 
system  requirements  like  monitors,  head-mounted  displays  and 
workstations/personal  computers  can  be  supplemented  with 
peripheral  combinations  of  CD-ROM,  laserdisk,  video  tape  and 
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other  external  storage  devices.  I/O  devices  consist  of  a 
keyboard  and  mouse  for  system  manipulation  and  a  joystick  for 
realistic  flight  simulation. 

Comparison  between  alternatives  allow  for  a  more 
comprehensive  cost/benefit  analysis. 
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III.  FRAMEWORK 


A.  INTRODUCTION 

Constructing  a  framework  for  evaluating  technology 
associated  with  this  desktop  CBT  will  require  a  incremental 
process  that  identifies  evaluation  criteria  and  then  applies 
those  criteria  to  the  functional  requirements.  This  chapter 
will  focus  on  the  evaluation  criteria  before  discussing  the 
actual  framework  construction.  This  framework  is  broad  in 
nature  and  allows  for  a  range  of  alternatives  to  be  presented. 

1.  Requirement  for  Effective  Analysis 

Although  the  system  that  is  being  proposed  in  this 
thesis  is  potentially  a  low-end  simulator,  it  is  imperative 
that  an  accurate  analysis  is  conducted  to  focus  the 
developmental  effort.  Because  many  computing  systems  may 
meet  the  criteria,  an  effective  evaluation  would  narrow  the 
choices  and  highlight  the  candidate (s)  that  best  meet  the 
requirements  of  the  users. 

2 .  Methodology 

The  evaluation  criteria  will  be  examined  to  help 
define  the  framework.  Consequently,  the  framework  will  be 
constructed  using  four  step  process  that  is  general  in  nature 
to  allow  for  across  platform  evaluation. 
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B.  FRAMEWORK 


1.  Evaluation  Criteria 

The  following  is  by  no  means  an  exhaustive  list.  It 
is,  however,  a  preliminary  set  of  measures  that  can  be  used  to 
analyze  a  CBT.  These  items  were  a  compilation  of  the 
collected  surveyed  information.  Each  criteria  is  discussed 
briefly  and  quantitative  measurement  factors  supplied,  when 
possible,  to  assist  in  the  evaluation  process, 
a .  Performance 

Computing  performance  consists  of  processing  and 
graphics  capability.  The  Central  Processing  Unit  (CPU) 
performs  the  critical  functions  of  any  computer  system, 
notably  the  processing  of  vast  amounts  of  instructions 
required  to  run  the  computer  applications.  The  unit  of 
measurement  of  the  CPU,  referred  to  as  clock  speed,  is  either 
megahertz  (MHz)  or  millions  of  instructions  per  second  (MIPS) . 

The  graphics  capability  is  crucial  to  overall 
system  performance.  Realistic  graphics,  displayed  either 
real-time  or  retrieved  from  system  storage,  should  be  a 
representative,  real-world  simulation.  As  previously  noted, 
only  the  users  can  determine  if  the  simulation  is 
representative . 

(1)  Processing  Capacity. 

•  Benchmarks :  In  order  to  test  system  performance  across 
different  platforms,  measurement  tools,  called  benchmarks, 
provide  quantitative  results,  normally  in  the  form  of 
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numerical  values.  The  higher  the  value,  the  better  the 
performance  of  each  respective  system  on  a  given 
application.  However,  care  must  be  exercised  when 
performing  benchmark  testing  or  using  benchmark  results. 
The  tested  platforms  must  be  of  similar  configurations 
and,  if  possible,  the  use  of  applicable  applications  and 
data  files  employed.  If  the  user  environment  cannot  be 
duplicated,  benchmark  measurement  could  be  inaccurate. 
Also,  benchmarks  are  sensitive  to  the  type  of  computing, 
e.g.,  integers  vs.  floating  point.  Normally,  single-user 
systems  can  be  effectively  benchmarked;  multi-user  systems 
are  more  difficult.  Industry  benchmarks  include 
Dhrystones,  Whetstones,  Khornerstones ,  Speclnt  and  SpecFP. 
[Ref.  6:  p.  3] 


(2)  Graphics  Capability . 


•  Refresh  Rate;  In  order  to  provide  a  realistic 
presentation,  it  is  generally  agreed  from  the  data 
collected,  that  the  refresh  rate  must  be  at  least  30  Hz 
(30  times/second) .  Faster  rates  are  preferable.  The 
lower  the  rate,  the  more  rigid  the  presentation  (i.e.,  the 
displayed  picture  "jerks") .  It  must  be  stressed  that 
refresh  rates  increase  with  a  smaller  displayed  "window" 
or  section  on  the  monitor.  In  other  words,  if  only  one 
quarter  of  the  monitor  were  to  display  the  simulation,  it 
would  refresh  at  a  higher  rate  than  if  the  entire  monitor 
was  being  used.  For  the  purposes  of  this  evaluation,  the 
full  monitor  presentation  is  recommended. 

•  Resolution:  Available  resolution  is  a  factor  of  the 

application  and  the  size  of  the  monitor.  The  larger  the 
screen  size,  the  better  the  resolution.  For  1280  x  1024 
pixel  resolution,  a  monitor  of  at  least  17  inches  is 
recommended  in  order  to  present  small  scale  detail.  Less 
resolution  is  required,  and  prescribed,  for  smaller 
monitors.  For  example,  a  14-inch  screen  looks  best  at  800 
x  600  pixels.  Another  factor  to  consider  is  dot  pitch. 
A  15-inch  screen  with  .31  mm  dot  pitch  is  equivalent  to  a 
14-inch  with  a  .28  dot  pitch.  This  is  because  the 
displayed  presentation  must  be  condensed  to  fit  the  width 
of  the  smaller  screen  and  making  the  higher  dot  pitch  and 
resolution  more  difficult  to  read.  [Ref.  7:  p.  14] 
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b .  Software  Support 

Applicable  software  must  be  identified  or  created 
in  order  to  perform  to  the  system  specifications.  Three  areas 
of  concern  are  presented. 


•  Availability  of  Existing  Night  Vision  Visual  Database: 
The  time-consuming  design  phase  of  the  visual  database 
would  be  precluded  if  a  representative  night-vision 
database  exists  and  is  available.  This  determination 
might  prove  difficult  because  of  the  lack  of  available 
information.  During  data  collection,  none  of  the 
individuals  surveyed  were  aware  of  an  existing  database. 


•  Tools  for  Database  Design:  Obviously,  if  one  is  not 

available,  then  the  visual  database  must  be  constructed. 
Developmental  tools,  such  as  MultiGen,  from  Software 
Systems  and  Replicore,  from  Kinetic  Visuals,  provide  three 
dimensional  model  building.  Other  tools  are  also 
available  to  perform  similar  functions. 

•  Access  to  DMA  and  USGS  Data;  In  order  to  create  real- 
world  scenarios,  rendering  of  Defense  Mapping  Agency 
charts  and  United  States  Geological  Survey  digital  mapping 
data  into  the  visual  database  is  required.  The  tools 
previously  identified  support  the  inclusion  of  the  digital 
data  in  database  construction. 


c.  Peripheral  Support 

The  system  must  be  able  to  support  a  number  of 
peripherals,  the  specifics  of  which  will  be  discussed  in 
Chapter  IV.  These  peripherals  will  augment  the  CBT  and 
increase  overall  system  capabilities  significantly. 


•  Interactive  I/O  Devices:  A  joystick  would  provide 

authentic  airborne  simulation.  Additionally,  a  keyboard 
and  mouse  would  provide  user  interface. 

•  Replay  Capability:  Training  would  be  maximized  if  the 

participating  aircrew  were  able  to  review  their 
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performance  and  note  any  deficiencies.  Additionally,  this 
debrief  capability  could  significantly  enhance  actual 
mission  preparation  by  providing  an  avenue  for  critique  of 
an  actual  mission  rehearsal. 

•  Support  of  a  Head-Mounted  Display.  Audio  and  Video  Input 
Devices :  These  devices  will  provide  the  realism  to  the 

system  simulation. 


d.  Migration  Path 

As  technology  continues  to  advance,  the  system  must 
be  capable  of  upgrading  itself  to  incorporate  greater 
capabilities  and  possibly  integrate  with  more  robust  systems. 

•  Reusability  of  Components:  Reusable  components  provide 
flexibility  and  cost  savings  and  reduce  the  need  for 
outside  support  requirements. 


e .  User  Needs 

Because  specific  user  requirements  are  not 
currently  available  and  are  subject  to  change,  a  general  list 
is  provided  to  help  provide  guidance. 


•  Ease  of  Use/Learning :  Because  of  the  possible  lack  of 

computer  literate  users,  the  system  simulation  must  be 
easily  accessible  with  a  tutorial  provided  to  expedite 
learning  and  training. 

•  Acceptable  MTBF :  Reliability  of  the  system  when  it  is 
deployed  is  a  relatively  important  issue.  A  mean-time- 
between-failure  value  must  be  determined  before  design  to 
ensure  the  appropriate  amount  of  "up  time"  is  provided  to 
the  aircrew. 

•  Manage  Specific  Operational  Profiles:  Because  of  the 

myriad  of  mission  specific  profiles  that  aircrew  must 
prepare  for,  the  system  must  have  available  at  least  a 
representative  simulation  of  prospective  geographic 
terrain. 
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f.  Management  Issues 


There  are  numerous  managerial  concerns.  The 
following  three  issues  are  perhaps  most  important  to  initial 
system  start-up. 


•  Availability  of  Clones:  If  warranted,  the  system  software 
needs  to  be  portable  across  multiple  hardware  systems  if 
cost/availability  issues  are  present. 

•  Vendor  Viability:  The  vendor  must  provide  the  services 
required. 

•  System  Mobility;  Directly  related  to  deployability,  the 
system  must  be  small  enough,  yet  resilient,  for  frequent 
relocation. 


g.  Head-Mounted  Display  Component  Capability 

The  actual  display  requirements  could  change  with 
maturing  technology.  The  following  address  existing  and 
future  growth  areas. 


•  Liquid  Crystal /Fiber  Optic:  Though  not  as  mature,  fiber 
optics  provide  better  throughput  and  growth  potential. 
Liquid  crystal  is  more  limited,  yet  prevalent. 

•  Refresh  Rate:  The  head-mounted  display  must  refresh 

itself  at  the  same  rate  as  the  computing  system. 

•  Field-of-View  (FOV) :  The  display  FOV  must  be  consistent 
with  the  night-vision  goggle  (NVG)  FOV  that  is  used  by  the 
aircrew. 

•  Monochrome /Color:  Because  a  night-vision  presentation  is 
in  shades  of  green,  a  color  capable  display  is  more 
realistic  because  it  will  support  the  different  green 
hues.  Monochrome  only  displays  shades  of  grey. 
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h.  Video  Compatibility 

Because  of  the  potentially  numerous  video  inputs  to 
the  system,  both  internal  and  external,  compatibility  with  the 
head-mounted  display  is  a  concern. 

•  In  Concert  with  Monitor:  The  head-mounted  display  must 
present  the  same  picture  as  the  system  monitor.  Training 
is  enhanced  with  viewing  by  multiple  aircrew  members. 

i.  Design  of  the  Head-Mounted  Display 

For  realistic  training  purposes,  the  physical 
characteristics  of  the  simulator  device  and  the  operational 
NVG  model  must  match. 


•  Size/Weiaht:  The  head-mounted  display  must  be  similar  in 
both  size  and  weight  to  the  NVG.  Additionally,  the 
display  must  be  robust  enough  to  withstand  frequent 
physical  transfer. 


2 .  Design  of  Framework 

Designing  this  evaluation  framework  required  a 
systematic  approach  be  utilized.  This  approach  followed  a 
four-step  process.  [Ref.  3:  p.38] 


1.  Identify  alternative  configurations 

2.  Define  functional  components 

3.  Tabulate  and  compare  capabilities  and  costs 

4.  Selection  Process. 
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The  alternative  configurations  were  narrowed  to  two 
candidate  systems,  a  workstation  and  a  personal  computer,  both 
broadly  defined,  to  preclude  restriction  in  functional 
capabilities.  In  a  realistic  environment,  more  alternative 
candidates  could  be  applied,  but  for  the  scope  of  this  thesis, 
only  these  two  were  selected. 

Defining  the  functional  components  was  a  direct  result 
of  reviewing  the  technology  available  by  gathering  applicable 
information  and  disseminating  the  data.  The  result  is  a 
component  list  that  is  somewhat  general  in  nature,  yet 
specific  in  capability. 

Tabulating  and  comparing  capabilities  and  costs  would 
help  identify  possible  fiscal  constraints  and  trade-offs  that 
would  naturally  result. 

The  selection  process  is  based  on  cost/ benefit 
measurements.  Overall  system  performance  must  be  measured 
against  costs  and  other  constraints. 

a.  Identify  Alternative  Configurations 

Two  hardware  designs  are  presented.  A  pc-based 
system  would  provide  a  low-cost  solution  with  a  possible 
decrease  in  overall  system  performance.  A  workstation-based 
system  delivers  the  computing  power  required  for  more 
realistic  simulation  with  a  significant  increase  in  cost. 
Both  alternatives  are  considered  because  they  offer 
cost/performance  alternatives. 
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b.  Define  Functional  Components 

At  this  point,  it  is  difficult  to  determine  exact 
functional  components.  However,  using  collected  data  from 
technical  experts  and  operational  users,  a  component  list  can 
be  generated  to  provide  specific  guidance  on  the  required 
components . 

c.  Tabulate  and  Competre  Capabilities  and  Cost 

Generating  tables  that  list  system  capabilities  and 

their  respective  costs,  grouped  by  function,  provide  direct 
access  in  order  to  compare  and  contrast  the  different 
alternatives . 

d.  Selection  Process 

This  process  is  based  on  selecting  the  alternative 
that  maximizes  benefit  or  value  and  still  fits  within  defined 
system  constraints.  The  selected  alternative  should  meet  the 
evaluation  criteria  to  make  it  representative  for  the  users. 
That  criteria  is  subject  to  change  once  the  user  requirements 
are  defined. 
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IV.  APPLICATION  OP  FRAMEWORX 


A.  PURPOSE 

This  chapter  presents  the  collected  information  of  the 
alternative  systems.  The  data,  which  is  a  review  of  current 
technology  commercially  available,  is  applied  to  the  framework 
presented  in  Chapter  III.  The  information,  though  cursory, 
will  give  the  reader  an  idea  of  current  computer  technology. 

B.  APPLIED  EXAMPLE 

l.  Identify  Alternative  Configurations 

a .  PC-Based 

With  the  growth  of  the  personal  computer  market, 
technological  capabilities  and  computational  power  of  the 
smaller,  less  sophisticated  personal  computer  has  increased 
dramatically.  With  the  proliferation  of  the  pc  and  related 
products,  and  the  resultant  competition  in  price,  economically 
viable  combinations  of  computer  and  peripherals  are  available. 
Figure  1  on  the  following  page  presents  a  possible  hardware 
configuration  that  could  deliver  the  prescribed  performance 
requirements . 
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Figure  1.  PC-Based  Configuration 
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b .  Works ta tion-Based 


Traditionally,  workstations  have  been  employed  for 
scientific  and  engineering  applications.  Their  processing 
power,  memory  capacity  and  displays  have  been  superior  to 
available  personal  computers.  This  technology  gap  has  been 
significantly  reduced  in  the  past  few  years,  but  the 
workstation  is  still  able  to  deliver  superior  graphical 
simulation.  Silicon  Graphics  has  captured  a  large  market 
share  and  is  the  industry  standard  in  graphics  presentation. 
Since  realistic  simulation  may  be  a  critical  element  of  this 
CBT,  the  workstation  option  must  be  seriously  considered. 

Most  available  workstations  have  standard  memory, 
speed  and  graphic  capabilities  that  meet  our  system 
requirements.  Figure  2  on  the  next  page  is  a  possible 
workstation-based  hardware  configuration. 
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Figure  2.  Workstation-Based  Configuration 
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2.  Define  Functional  Components 
a.  Personal  Computer 

(1)  Component  Architecture.  Available  components 
include  an  Intel  486  chip  with  a  co-processor  for  performing 
complex  mathematical  functions;  an  internal  disk  drive  with  at 
least  250  megabytes  of  storage;  a  dedicated  graphics 
processor/ accelerator  with  24-bit  color,  two  megabytes  of 
video  RAM  (VRAM)  to  provide  dedicated  graphics  processing  and 
supply  a  refresh  rate  of  at  least  20-30  Hz,  full  screen;  a 
256k  processor  cache  to  assist  CPU  processing;  a  video  board 
that  can  capture  (access)  video  from  a  variety  of  sources 
(i.e.,  8mm  video  tape,  laserdisk,  super  VHS)  and  retrieve  both 
real-time  and  from  storage;  a  32-bit  VESA  local  bus  (VL-Bus) 
to  supplement  the  graphics  accelerator;  and  an  audio  card  that 
is  compatible  with  audio  CD\ROM  and  digital  audio  tapes  (DAT) . 

Three  available  CPUs  for  a  486  computer  are  DX- 
33  MHz,  DX-50  MHz  and  DX2-66  MHz. 

On  initial  examination,  it  would  appear  that 
the  66  MHz  clock  speed  is  most  desirable.  However,  because  it 
is  a  DX2  configuration,  the  CPU  runs  at  66  MHz,  but  the  mother 
board,  which  performs  the  applications,  runs  at  only  33  MHZ. 
In  other  words,  the  computing  instructions  are  collected  and 
performed  at  66  MHz,  while  the  data  transfer  is  accomplished 
at  33  MHz. 
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The  50  MHz  machine  has  a  CPU  and  a  motherboard 
that  both  perform  at  50  MHz.  This  allows  more  uniform 
processing  in  computing  functions  and  an  overall  increase  in 
performance . 

Eight  to  16  megabytes  of  random  access  memory 
(RAM)  should  provide  the  necessary  memory  space  required  by 
the  system.  Additionally,  an  operating  system  that  can  access 
this  extended  memory,  such  as  IBM's  OS-2  2.0  or  Microsoft's 
Windows  NT,  would  optimize  memory  storage  and  provide  for  an 
enhanced,  multi-media  presentation. 

(2)  Monitor.  A  high-resolution,  non-interlaced 
(reduced  flicker),  17",  SVGA  monitor  is  prescribed.  It  can 
support  a  resolution  of  up  to  1200  x  1024  pixels  and  up  to 
16.7  million  colors,  though  not  in  combination.  A  larger, 
flat  screen  monitor  that  reduces  the  amount  of  glare  is  also 
preferable,  though  not  required.  The  resolution  and  refresh 
rate  may  be  critical  for  realistic  simulation.  Most  can 
support  a  refresh  rate  of  at  least  60  Hz,  depending  on  screen 
window  size. 

(3)  Head-Mounted  Display.  The  display  must  refresh 
at  the  same  rate  as  the  monitor,  for  presentation  continuity. 
The  size  and  weight  should  be  representative  of  the  night- 
vision  goggles  currently  employed  operationally,  with  color 
capability  and  similar  f ield-of-view  (FOV) . 
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(4)  Peripherals .  There  are  a  number  of  peripherals 
that  may  provide  the  required  performance.  The  following 
discussion  will  focus  on  memory  storage  and  audio  and  video 
inputs . 

A  12"  optical  laserdisk  which  normally  has  a 
capacity  of  five  gigabytes  of  data  storage  should  supply  the 
required  external  memory  requirements.  Write-once,  read-many 
(WORM)  technology  allows  for  specific  data  mastering  and 
storage,  and  retrieval  when  necessary. 

Video  input  sources  to  the  simulation  include 
8mm,  video  laserdisk,  VHS  and  CD\ROM.  A  video  capture  board 
that  is  compatible  with  all  of  these  sources  would  be 
required.  Most  have  the  capability  of  accessing  the  video 
real-time  and  retrieving  compressed,  stored  data  from  an 
internal  digital  file. 

Audio  input  sources  include  both  audio  CD\ROM 
and  digital  audio  tape  (DAT) .  They  afford  greater  memory 
storage,  smoother  signal  retrieval  with  reduced  audio 
distortion. 

b.  Workstation 

(1)  Component  Architecture.  CPU  speeds  range  from 
33  to  150  MHz.  These  speeds  should  produce  the  required 
processing  power,  depending  on  the  application.  Obviously,  a 
higher  speed  is  preferable.  Internal  memory  of  16  megabytes 
and  a  hard  disk  size  of  3  00  megabytes  are  somewhat  standard  on 
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most  workstations  and  are  appropriate  baselines  for  delivering 
required  performance. 

Graphical  presentation  are  often  measured  in 
polygons/second  in  a  10  by  10  pixel  box,  triangle  mesh  in  a 
five  by  10  pixel  box,  both  in  full,  24-bit  color,  and  three- 
dimensional  vectors.  Baseline  numbers  are  difficult  to 
determine  because  the  values  decrease  as  the  screen  size 
increases.  Comparisons  must  be  made  on  equivalent  screen 
sizes.  Full  screen  refresh  normally  at  least  30  Hz, 
depending  on  window  size  and  application. 

(2)  Monitor.  A  minimum  size  of  17"  would  take 
advantage  of  the  resolution  capabilities  of  most  workstations. 
They  are  available  with  24-bit  color  (224  number  of  colors)  and 
1280  by  1024  pixels. 

(3)  Head-Mounted  Display.  Requirements  are  the 
same  as  for  the  PC.  The  displays  that  were  surveyed  had 
connection  capabilities  with  both  personal  computers  and 
workstations . 

(4)  Peripherals.  The  peripherals  described 
previously  in  the  discussion  of  personal  computers  are  also 
applicable  to  the  workstation  configuration. 

3.  Tabulate  and  Compare  Capabilities  and  Cost 

Two  tables  have  been  constructed.  Appendix  A  contains 
the  information  on  personal  computers  while  Appendix  B 
contains  the  information  for  workstations. 
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These  tables  are  the  foundation  for  a  cost/benefit 
analysis  that  is  completed  in  the  next  step.  Price  ranges 
and  baseline  capabilities  are  provided,  and  specific  products 
are  described  periodically  to  help  the  reader.  For  instance, 
the  head-mounted  display  chosen  is  the  Kaiser  Electro-Optics 
Sim  Eye  40  and  for  the  workstation,  the  Silicon  Graphics  Iris 
Indigo  XL. 

a.  Configuration  Compeurison 

Comparing  the  two  configurations,  we  can  refer  to 
Appendix  A  and  B  for  baseline  capabilities  and  prices.  The 
price  ranges  provided  for  the  PC  system  are  rough  estimates 
based  on  numerous  brand  names.  Various  discounts  are  often 
available  for  bulk  purchase  and  the  Government  Service 
Administration  (GSA)  provides  specific  pricing  for  products 
from  many  vendors. 

The  Iris  Indigo2  XL  lists  at  $17,995  [Ref.  8:  p.l]. 
Discounts  are  available.  For  example,  the  Naval  Postgraduate 
School  receives  a  35%  reduction  in  list  price  for  this  unit. 

It  has  a  base  memory  is  32  megabytes,  a  CPU  clock 
speed  of  100  MHz  and  a  hard  disk  size  of  1  gigabyte. 
Additionally,  a  17  inch,  24-bit  color  monitor  is  provided  in 
the  price.  The  graphics  capabilities  include  1280  by  1024 
resolution  and  a  maximum  refresh  rate  of  60  Hz. 

Prices  of  personal  computers  varies  greatly  among 
commercial  vendors.  For  a  486-50  MHz  machine  with  16 
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megabytes  of  RAM,  a  hard  disk  size  of  250  megabyte  and  a  VL- 
bus  can  cost  between  $1,400  and  $2,000.  Audio  and  video 
boards  vary  greatly,  depending  on  performance  qualities. 
Audio  boards  for  CD\R0M  and  DAT  can  cost  between  $200-$400  and 
video  boards  that  are  able  to  capture  inputs  from  various 
sources  range  from  $300-$l,000.  Additionally,  a  dedicated 
graphics  processor/accelerator  can  range  in  price  from  $200- 
$900.  A  17  inch,  non-interlaced  monitor  that  can  support  1280 
by  1024  pixels  can  cost  $1,250. 

Prices  for  peripherals  also  vary.  Once  specific 
performance  requirements  are  determined,  a  more  definite  price 
structure  can  be  designed. 

As  described,  the  workstation  performance 
capabilities  exceed  the  pc.  The  trade-off  is  cost.  Though 
cheaper,  the  pc  must  be  provided  with  add-on  features  such  as 
video  and  audio  boards. 

Reliability  is  a  factor.  Name-brand  personal 
computer  vendors  (i.e.,  IBM,  Dell,  Compaq)  tend  to  supply 
better  warranties  and  readily  available  maintenance 

facilities.  Many  workstation  vendors  are  very  established  in 
the  market  and  pride  themselves  on  performance  dependability. 
b .  Software  Concerns 

In  order  to  address  the  software  concerns  evident 
in  the  evaluation  criteria,  a  very  simplistic  software 
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configuration  design  is  offered  to  help  describe  the 
correlation  between  the  subsystems. 

The  authoring  system  is  the  software  integrator. 
It  directly  receives  inputs  from  the  users  and  distributes 
those  commands  to  the  various  subsytems.  It  provides 
diagnostic  capabilities  and  supports  window  managers  and 
managerial  assistance  tools  such  as  expert  systems  and 
artificial  intelligence. 

Hypertext  acts  as  a  linking  mechanism  for  the  audio 
and  video  control  software.  It  provides  a  "point  and  click" 
function  to  assist  users  in  accessing  various  information  such 
as  tutorials  and  help  menus. 

The  DBMS  manipulates  the  database,  when  directed  by 
the  authoring  system,  and  provides  the  appropriated 
information  to  the  simulation  software. 

The  simulation  software  operates  with  the  graphics 
generation  software  to  deliver  the  prescribed  presentation 
requested  by  the  authoring  system.  This  presentation  is  the 
displayed  simulation. 

Figure  3  on  the  following  page  depicts  the  software 

design. 
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Figure  3.  Software  Configuration 


The  building  of  a  representative  database  or 
employment  of  an  existing  one,  must  be  accurate  and  the 
resultant  graphics  generated,  displayed  and  updated  quick 
enough  to  provide  the  required  realism. 

A  basic  software  summary  table,  without  any 
representative  values,  is  contained  in  Appendix  C.  It  has 
been  designed  to  apply  some  of  the  evaluation  criteria  and 
assist  in  the  selection  process. 

4.  Selection  Process 

a.  Capabilities  Versus  Specifications 

The  two  alternative  configurations  are  capable  of 
meeting  the  current  specifications. 

As  previously  discussed,  the  workstation  generally 
has  greater  display  capability,  but  a  PC  can  present  a 
graphical  display  that  may  prove  more  than  adequate.  Until 
reviewed  by  the  users,  both  may  be  capable  of  delivering  the 
required  realism. 

Because  of  the  peripheral  storage  space  available 
to  both  configurations  and  the  internal  memory  capacity,  a 
night  vision  database,  once  developed,  could  be  integrated 
into  the  personal  computer  and  the  workstation.  Until  then, 
a  video  source  would  have  to  be  supplied,  such  as  full-motion 
video.  Both  alternatives  support  this  option. 

The  alternatives  presented  are  compatible  with  the 
head-mounted  display  used  in  the  practical  example  (the  Sim- 
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Eye  from  Kaiser  Electro-Optical).  It  can  be  assumed  that 
other  head-mounted  displays  will  also  be  compatible. 


Transportation  of  the  two  configurations  can  be 
accomplished  with  very  little  effort.  However,  care  must  be 
exercised  when  packing  and  moving  the  head-mounted  display. 
The  Sim-Eye  was  relatively  fragile  and  would  require  detailed 
packing  procedures  before  transport. 

Finally,  both  configurations  meet  the  constraints 
of  system  delivery  and  complexity.  They  can  be  tailored  to 
meet  specific  user  requirements  and  are  currently  available  on 
the  commercial  market. 

However,  until  defined,  cost  will  remain  a  variable 
in  the  selection  process.  As  noted  earlier,  the  workstation 
configuration  cost  more  but  delivers  greater  performance. 
Once  a  cost  figure  is  determined,  a  final  judgement  can  be 
made  on  this  constraint. 

b.  Evaluation  Criteria 

Applying  the  evaluation  criteria  to  the  identified 
alternatives  and  in  conjunction  with  the  comparison  tables, 
will  ensure  that  the  system  selected  maximizes  benefit  and 
minimizes  costs  and  risk.  The  specifications  described  in  the 
criteria,  though  not  rigid,  provide  a  reference  for  the 
selector(s)  to  recognize  the  most  appropriate  candidate. 
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c.  Subjective  Measurements 

While  cost  is  an  extremely  objective  measure,  other 
factors  are  not.  Both  benefit  and  risk  sometimes  require 
subjective  measurements.  What  might  be  beneficial  for  one 
group  of  users  is  not  for  others.  In  order  to  reach  a 
consensus  among  users,  subjective  factors  must  be  quantified, 
at  least  partially.  Assigning  relative  weight  factors 
sometimes  enable  the  selection  process  to  become  more 
objective.  Although  not  always  successful,  it  should  be 
attempted . 

C.  SUMMARY 

This  example  has  given  some  basic  guidelines  to  follow  in 
order  to  apply  the  framework.  Though  cursory  in  nature,  this 
illustration  helped  to  demonstrate  the  ease  of  using  the 
straightforward  four-step  process.  An  actual  selection 
determination  was  not  provided  because  of  uncertain  user 
requirements,  but  the  tabulated  information  helped  to  display 
the  various  performance  capabilities  that  currently  exist,  as 
well  as  their  associated  costs.  A  direct  comparison  between 
alternatives  help  to  clarify  some  of  the  pertinent  issues  that 
the  applicable  decision  makers  must  resolve. 
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V.  CONCLUSIONS  AND  RECOMMENDATIONS 


A.  REVIEW 

1.  Design  Concerns 

Chapter  II  provided  insight  into  some  of  the  issues 
related  to  this  system.  Uncertain  user  requirements,  possible 
specification  changes  and  short  developmental  time-frame 
restrictions  must  all  be  addressed.  A  review  of  current 
simulator  techniques,  computer  technology  trends  and  the 
emphasis  on  off-the-shelf  procurement  provided  elaboration  to 
the  discussion.  Finally,  a  methodology  was  described  and 
detailed.  It  followed  a  progression  from  specification 
definition  to  possible  alternative  design. 

2 .  Framework 

Chapter  III  introduced  the  framework  and  described 
possible  evaluation  criteria.  The  list  of  criteria  entails 
revision  as  technology  and  requirements  change.  Ihe  framework 
consist  of  four  basic  steps: 

1.  Identify  Alternative  Configurations 

2.  Define  Functional  Components 

3.  Tabulate  and  Compare  Capabilities  and  Cost 

4.  Selection  Process 
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3.  Application  of  Framework 

Chapter  IV  presented  an  application  of  the  framework. 
Collected  data  on  both  personal  computers  and  workstations  was 
presented.  This  information  contained  target  technical 
capabilities  and  cost  ranges.  The  example  clarified  how  the 
framework  could  be  applied  to  make  qualitative  decisions  on 
alternative  systems  and  assist  in  the  selection  process. 

B .  FUTURE  ENHANCEMENTS 

There  are  many  computer  components  and  capabilities  that 
could  potentially  supplement  this  CBT.  Three  possible 
enhancements  are  discussed  and  they  all  may  be  available  for 
integration  in  the  system  development. 

1.  Pentium 

Noted  earlier,  personal  computers  are  continuing  to 
bridge  the  technology  gap  with  workstations.  The  Pentium 
processor,  rated  at  either  60  or  66  MHz,  employs  much  faster 
processing,  in  some  cases  by  a  factor  of  10.  Consequently, 
any  personal  computer  decision  must  include  compatibility  with 
the  Pentium  for  upgrade  possibilities.  Estimated  cost  for 
adding  the  Pentium  chip  is  about  half  the  cost  of  purchasing 
a  new  Pentium  system. 

2.  RISC 

An  additional  upgrade  for  the  personal  computer  is  the 
employment  of  the  reduced  instruction  set  chip  (RISC) . 
Currently  employed  in  most  workstations,  the  RISC  processes 
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the  most  important  instructions,  instead  of  all  the 
instructions,  thus  reducing  access  time,  increasing  the  data 
transfer  rate  and  significantly  increasing  overall  system 
performance.  OS-2  2.0  and  Windows  NT  work  across  both  CISC 
and  RISC  platforms. 

3 .  Networking 

Although  designed  to  be  a  stand-alone  system  when 
deployed,  this  desktop  CBT  could  be  connected  to  other 
available  systems,  if  feasible.  This  would  maximize  training 
for  multiple  users  by  providing  an  avenue  for  improving 
f light-lead/wingman  discipline. 

4.  Improvements  in  Head-Mounted  Displays  and  Display 

Technology 

As  technology  continues  to  develop,  improvements  to 
head-mounted  displays  will  be  realized.  Upgraded  capabilities 
in  the  areas  of  increased  f ield-of-view  and  resolution  will 
provide  the  aircrew  with  state-of-the-art  simulation.  Head- 
mounted  devices  that  integrate  all  visual  cues,  whether  day  or 
night,  are  currently  in  development  and  virtual  reality 
displays,  in  the  form  of  wrap-around  goggles,  are  potential 
future  components. 
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C.  RECOMMENDATIONS 


1.  User  Requirements 

Operating  this  system  will  directly  challenge  the 
participating  aircrew.  Many  have  never  operated  even  the  most 
rudimentary  computer  system  and  those  that  have  usually 
restrict  their  usage  to  word  processing  applications.  Heavy 
emphasis  must  be  placed  on  gathering  user  requirements. 
Without  their  direct  input  in  the  design  and  implementation 
phases,  overall  system  failure  may  result  due  to  lack  of 
usage.  Naval  aviation  traditionally  prides  itself  on  its 
ability  to  overcome  and  master  any  environment.  However,  most 
aviators  devote  little  time  to  projects  that  are  designed  and 
developed  without  their  direct  user  input. 

Elements  requiring  special  attention  include  ease  of 
use/training;  a  low  mean  time  between  failure  (MTBF) ;  a  high 
quality  of  vendor  support,  especially  in  the  initial  delivery; 
and  the  ability  to  tailor  the  simulation  to  specific 
operational  profiles. 

Once  the  user  requirements  have  been  determined,  a 
comprehensive  cost\benefit  analysis  can  be  conducted  utilizing 
the  framework  previously  discussed.  Using  the  possible 
baseline  values  presented  in  our  analysis,  commercial  vendors 
can  be  contacted  and  prices  negotiated  for  military  and/or 
bulk  discounts.  The  current  emphasis  on  procuring  of  off-the- 
shelf  products  lends  itself  to  our  benefit.  This  CBT  could  be 
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designed,  prototyped  and  refined  quickly,  making  it  available 
to  operational  units  in  less  time. 

2.  Continual  Framework  Revision 

As  computer  technology  continues  to  advance,  it  is 
extremely  important  that  the  prescribed  framework  be  revised 
to  fit  the  technological  growth.  The  inherent  simplicity 
allows  for  review  and  amendment.  Since  it  is  operating  in  a 
highly-dynamic  environment,  the  framework  must  respond  to  the 
intrinsic  progression  of  computer  technology  if  it  is  going  to 
remain  a  viable. 

D.  CONCLUSIONS 

There  is  no  doubt  that  the  hardware  exists  for 
constructing  this  desktop  CBT.  The  lack  of  a  representative 
night  vision  database  may  cause  delay  in  software  design,  but 
other  means  can  be  employed  to  at  least  provide  a  rudimentary 
presentation  until  a  realistic  database  could  be  integrated. 
The  guidelines  discussed  have  presented  a  framework  to  gather 
specific  information  essential  in  comparing  various 
technologies.  The  amount  of  commercial  vendors,  easily 
accessible  literature  and  an  express  desire  among  vendors  to 
participate  in  defense  programs  combine  to  make  this  CBT 
extremely  viable. 

It  would  be  easy  to  say  that  cost  is  the  most  restrictive 
factor.  If  that  were  true,  then  the  cheapest  alternative, 
with  minimum  capabilities  would  be  selected.  That,  however. 
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should  not  be  the  case.  Concern  about  future  upgrade 
capability  is  a  valid  one  and  often  less  sophisticated 
(cheaper)  systems  have  little  or  no  upgrade  options.  However, 
realistic  pricing  concerns  must  be  addressed  because  of  the 
fiscal  climate  that  exists  within  the  Department  of  Defense, 
especially  in  the  area  of  procurement. 

Other  methods,  outside  of  cost,  exist  to  make  an 
evaluation  in  preparation  for  acquisition.  Overall  technical 
excellence,  detailed  scheduling  and  delivery  plans  and  quality 
management  all  have  attractive  benefits.  Combining  all  these 
methods,  including  cost,  would  allow  for  a  comprehensive 
approach  to  development,  implementation  and  maintenance. 

The  question  of  "how  much  is  enough"  is  difficult  to 
answer.  Because  this  system  has  yet  to  be  constructed  and 
certain  design  requirements  are  uncertain,  especially  night- 
vision  database  design,  it  is  hard  to  set  specific  performance 
standards.  What  we  have  done  is  set  baseline  criteria  that 
provide  preliminary  performance  goals.  The  two  alternative 
configurations  presented  both  have  attractive  features.  While 
the  PC  is  less  costly,  it  performs  at  a  reduced  rate  compared 
to  the  workstation.  However,  the  workstation  is  more 
expensive,  though  perhaps  more  reliable.  Choosing  between  the 
two  will  require  determining  trade-offs  in  performance,  cost, 
reliability  and  maintenance.  All  these  variables  must  be 
factored  into  the  final  decision. 
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What  is  most  appealing  about  the  desktop  CBT  concept  is 
that  it  offers  a  less  costly  alternative  to  large  scale 
simulation.  Whether  PC  or  workstation-based,  it  has  unique 
deployability  features  and  it  offers  training  opportunities 
where  none  have  previously  existed. 

If  naval  aviation  is  going  to  survive,  it  must  maximize 
every  opportunity  to  train,  especially  while  deployed.  What 
better  way  to  comply  than  to  develop  a  well-designed  and 
upgradable  desktop  CBT;  one  that  prepares  aircrew  for  the 
labor-intensive  and  dynamic  mission  of  low-level  navigation 
employing  night  vision  goggles. 
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APPENDIX  A:  TABULATED  INFORMATION  FOR  PC-BASED  8YSTEM 


cost:  $1,400-52,000 


RAM:  16  MB 


CPU:  50  Mhz 


VL-Bus:  Yes 


Processor  Cache:  256  K 


Hard  Disk  Size:  250  MB 


Cost:  $200-$400 


Compatible  w/ CD-ROM  &  DAT:  Yes 


Cost:  $1000 


Capture  Elements:  8mm, CD\ROM, SVHS, LaserD. 


Cost:  $2  00-$900 


Memory:  2  MB  Video  RAM  (VRAM) 


Bit  Color:  24 


3-D?:  Yes 
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APPENDIX  A  (Cont.) 


PERIPHERALS 


Monitor 


Cost:  $1,250 


Size:  17  inch 


Interlaced/Non-interlaced:  NI 


Resolution:  1280  by  1024 


Available  Colors:  256  (full  screen) 


Head-Mounted 

Display 

Cost:  $95,000;  Bulk  discounts  available 

LCD/ Fiber  Optic:  LCD 

Field  of  View  (FOV) :  40  V  by  60  H  degree 

Refresh  Rate:  60  Hz  maximum 

.  .  .  1 

Monochrome / Color:  Both 

Video  Input 
Device 

Type:  VCR 

Type:  Laserdisk 

Cost:  $300-$700 

Cost:  $400-$l , 200 

Audio  Input 
Device 


Type:  CD \ ROM 

Type :  DAT 

Cost:  $2  00-$500 

Cost:  $300-$500 

Storage  Disk 


Cost:  $4 , 000-$6, 000 


Memory :  5  GB 


50 


APPENDIX  B:  TABULATED  INFORMATION  FOR  WORKSTATION-BASED 

SYSTEM 


Workstation 


Graphics 

Capability 


Peripherals, 
Excluding  Monitor 


Cost:  $17,995  (List) 


SpecInt92/SpecFP92 :  58.6/60.6 


Base  Memory  Size:  16  MB 


Hard  Disk  Size:  1  GB 


UNIX  Compatible:  Yes,  IRIX  supplied 


Full  Screen 
Refresh  Rate:  60  Hz 


3-D  Vectors:  490  K 


Polygon/ Sec:  60  K 


T  Mesh:  40  K 


Cost:  Included  in  workstation  price 


Size:  17  inch 


Colors  (Bits) :  24 


See  Appendix  A 
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APPENDIX  C:  POS8IBLE  SOFTWARE  REQUIREMENTS 


Software 

Support 

Authoring  Tool: 

Cost : 

Diagnostic  Capabilities: 

Window  Manager  Support?: 

Support  for  AI/ Expert  System  Shells?: 

Visual  Database  Tool: 

Cost: 

Access/Rendering  of  DMA/USGA 

Terrain  Data?: 

Symbology  Tool: 

Cost: 
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